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EXECUTIVE  SUMMARY 


The  Civil  Aerospace  Medical  Institute  (CAMI),  as  a  com¬ 
ponent  of  the  Office  of  Aerospace  Medicine  (OAM),  surveyed 
the  population  of  aviation  medical  examiners  (AMEs),  as  federal 
designees,  in  2012  to  assess  their  satisfaction  with  Federal  Avia¬ 
tion  Administration  (FAA)  airman  medical  certification  services 
and  to  gather  their  feedback  on  how  to  improve  those  services. 
Needed  service  improvements  identified  by  survey  respondents 
were  captured  several  ways,  including  multiple-choice,  ranking, 
and  open  text  items.  The  format  allowed  for  the  collection  of  a 
wealth  of  data  in  terms  of  detail  and  volume. 

Reported  here  are  results  from  an  in-depth  content  analysis 
of  AME  recommendations  for  improving  1 0  specific  medical  cer¬ 
tification  service  areas,  processes,  and  policies.  Recommendations 
made  by  U.S.,  civilian  AMEs  were  analyzed  separately  from  the 
military,  federal,  and  international  (MFI)  AMEs  recommenda¬ 
tions,  not  for  the  purpose  of  comparison  but  due  to  different 
operational  conditions.  Results  provide  a  programmatic  view  of 
AME-recommended  improvements  and  can  be  used  to  inform 
future  OAM  decisions  regarding  medical  certification  services. 

One  in  five  AMEs  (438  of  2,118:  392  domestic  and  46 
MFI)  that  met  the  survey  selection  criteria  responded  to  at  least 
one  of  the  open  text  items.  Of  those,  most  reported  being  a 
senior  AME  (75%  domestic,  81%  MFI).  Half  of  the  domestic 
respondents  reported  at  least  20  years  of  experience  as  an  AME 
and  conducting  at  least  78  exams  in  the  previous  12  months 
(range  3-3,800).  Half  of  the  MFI  respondents  reported  at  least 
1 5  years  of  experience  as  an  AME  and  conducting  at  least  30 
exams  in  the  12-month  period  (range  1-300).  All  domestic  and 
international  regions  were  represented  by  the  respondents.  Of 
note,  the  majority  of  those  providing  recommendations  who 
had  used  the  medical  certification  service  in  the  past  12  months 
reported  satisfaction  with  the  service. 


Domestic  AMEs  provided  794  recommendations  and  the 
MFI  group  provided  101.  Actionable  feedback  categories  emerged 
based  on  common  content  identified  during  the  initial  phases 
of  the  content  analysis.  The  feedback  categories  converged  on 
three  main  areas  for  improvement,  which  were  to  develop  orga¬ 
nizational  capability,  enhance  digital  systems/ tools,  and  change 
medical  certification  processes/policies.  Ninety-six  percent  of 
the  895  recommendations  were  classified  under  one  of  the  main 
areas,  with  nearly  twice  as  many  recommendations  pertaining  to 
development  of  organizational  capabilities,  compared  to  the  latter 
two  areas.  The  4%  of  recommendations  excluded  from  the  final 
ph  ase  of  the  content  analysis  were  classified  as  miscellaneous. 

The  actionable  feedback  for  the  three  high-level  areas  fol¬ 
lows  in  order  of  priority,  i.e.,  largest  to  smallest  frequency.  AME 
feedback  regarding  development  of  organizational  capability 
included  need  for:  training,  real  time/anytime  access  to  FAA 
physicians,  specific  information,  timely  communication  and  spe¬ 
cific  content  in  correspondence,  speedier  FAA  decisions,  reduced 
costs,  quality  interactions  with  FAA  personnel,  and  an  increase 
in  FAA  staff.  Feedback  regarding  enhancements  to  systems/tools 
included  need  for:  new  capabilities,  ready  and  stable  access,  end 
user  support  in  effective  and  efficient  task  performance,  easier  to 
use  and  read  interfaces  and  printouts,  and  adjustment  to  exist¬ 
ing  capabilities.  Finally,  feedback  regarding  changes  to  medical 
certification  processes  and  policies  addressed  requirements  for: 
transmission  of  reports,  records,  and  documents  to  the  FAA; 
airmen  applications;  exam  appointments;  issuance  decisions; 
printed  certificates;  and  AME  rules.  Some  of  the  recommended 
improvements  may  not  be  feasible  due  to  operational,  financial, 
or  regulatory  constraints. 


Aviation  Medical  Examiner  (AME)  2012  Feedback  Survey: 
Content  Analysis  of  Recommendations 


I.O  INTRODUCTION 

The  Civil  Aerospace  Medical  Institute  (CAMI),  as  a  compo¬ 
nent  of  the  Office  of  Aerospace  Medicine  (OAM),  periodically 
surveys  the  population  of  Aviation  Medical  Examiners  (AME)  to 
assess  their  satisfaction  with  airman  medical  certification  services 
and  to  gather  their  feedback  on  how  to  improve  those  services. 
Survey  respondents  provide  feedback  via  multiple  types  of  items, 
to  include  open  text.  Reported  here  are  the  analytic  approach  and 
results  from  an  in-depth  content  analysis^  conducted  on  the  open 
text  recommendations  for  improving  agencies’  services,  digital 
systems/tools,  training,  and  certification  processes  and  policies. 
The  results  provide  a  programmatic  view  of  AME-identified 
improvements  and  frame  results  in  actionable  terms. 

Background  on  the  AME  survey  is  provided  in  this  section. 
Details  of  the  content  analysis  procedure  and  results  follow, 
respectively,  in  Sections  2.0  and  3.0.  A  discussion  of  the  results 
is  presented  in  Section  4.0. 


overall  satisfaction  of  AMEs  with  services  and  quality  aspects  of 
services  provided  by  OAM  components  [i.e..  Aerospace  Medical 
Education  Division  (AMED),  Aerospace  Medical  Certification 
Division  (AMCD),  and  Office  of  the  Regional  Flight  Surgeon 
(RFS)]  that  they  may  request  or  receive  support  from.  The  sur¬ 
vey  also  captured  AME  feedback  on  improvements  to  offered 
services,  training,  the  deferral  process,  standards  and  guidelines 
for  medical  certification,  and  tools  (i.e.,  digital  electrocardiogram 
(ECG)  system,  Internet-based  Aerospace  Medical  Certification 
Subsystem  (AMCS),  MedXPress,  and  OAM  website^).  Survey 
items  on  training  covered  a  3 -year  period;  all  other  items  covered 
a  1 2-month  period,  with  the  exception  of  the  demographic  items. 

1.4  Survey  Structure  and  Item  Types 

The  survey  design  allowed  an  AME  to  skip  irrelevant  items. 
For  instance,  if  an  AME  reported  no  interactions  with  an  agency 
in  the  past  12  months,  then  no  additional  satisfaction  or  feed¬ 
back  data  on  the  agency  were  collected.  Thus,  the  number  of 


Table  1 .  Service  providers  and  services  presented  for  open  text  recommendations 


1.  AMED 

7.  Digital  ECG  System 

2.  AME  Training 

8.  AMCS  Internet  System 

3.  AMCD 

9.  MedXPress 

4.  RFS  Office 

10.  Online  Information  and  Publications 

5.  Deferral  Process 

1 1 .  Other  Recommendations 

6.  Standards  and  Guidelines 

1.1  Purpose  of  the  Survey 

Survey  results  are  used  by  the  OAM  to  improve  services 
provided  to  AMEs.  Administration  of  the  survey  meets  federal 
requirements  set  forth  initially  by  Executive  Order  No.  12862, 
“Setting  Customer  Service  Standards,”  and  the  Government 
Performance  and  Results  Act  of  1993. 

1.2  Selection  Criteria  for  Survey  Respondents 

In  2012,  the  survey  was  administered  to  all  active  AMEs — 
domestic,  military,  federal,  and  international.  An  invitation  to 
participate  from  OAM  was  sent  to  3,393  AMEs.  However,  only 
data  from  respondents  who  report  having  served  as  a  designee 
for  at  least  a  year  and  examined  at  least  one  airman  applicant 
for  certification  within  the  previous  12  months  were  included 
in  the  content  analysis. 

1.3  Survey  Content 

The  survey  items  were  developed  to  provide  both  general 
and  specific  feedback  to  the  OAM.  Survey  items  measured  the 

^  Content  analysis  is  a  data  reduction  technique  used  to  systematically  and 
objectively  extract  meaning  of  text  entries,  thereby  converting  qualitative  data 
into  quantifiable  terms  (see  U.S.  General  Accounting  Office,  1996). 


items  presented  to  an  AME  depended  on  that  AME  s  exposure 
to  different  agencies  over  the  evaluation  period.  Those  who  met 
the  selection  criteria  (described  next)  had  the  option  to  respond 
to  at  least  41  of  107  possible  items. 

The  survey  was  made  up  of  various  types  of  items — single¬ 
choice  (e.g.,  yes/no,  satisfaction  rating  scales,  demographic  cat¬ 
egories),  ranking  (e.g.,  indicate  top  3),  mark  all  that  apply  (e.g., 
reasons  why  and  needed  improvements),  numeric  write-ins  (e.g., 
percentages,  number  of  years)  and  open  text  (e.g.,  explanation 
of  “other,”  recommendations).  The  final  survey  item  asked  for 
additional  recommendations,  beyond  what  was  already  provided 
in  the  ranking  questions,  on  improving  airman  medical  certifica¬ 
tion  services  (Table  1). 

2.0  Analytic  Approach 

A  phased-approach  was  used  in  the  content  analysis.  The 
overarching  goal  was  to  convert  the  qualitative  data  collected 
with  the  open  text  items  into  quantitative  data  by  coding  it  into 
meaningful  actionable  feedback  categories  relevant  to  the  AME 
program  goals.  With  that,  the  following  phases  were  conducted: 
(1)  preparation  of  the  datasets  and  conduct  of  an  independent 


^  http://www.faa.gov/ about/ office_org/headquarters_offices/ avs/ offices/ aam/ 
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Table  2.  Counts  for  domestic  respondents,  and  coded  responses  and  comments 


AMED 

AME 

Training 

AMCD 

RFS 

Office 

Deferral 

Process 

Standards 

& 

Guidelines 

EGG 

System 

AMCS 

Med- 

XPress 

Online 
Info  & 
Pubs 

Other 

Respondent 

16 

91 

39 

69 

49 

73 

53 

62 

110 

39 

103 

Coded 

Response 

18 

125 

64 

70 

31 

60 

62 

67 

69 

57 

171 

Coded 

Comment 

4 

18 

7 

41 

6 

6 

9 

12 

39 

17 

21 

verification  and  validation  (V&V)  of  the  data;  (2)  identification 
of  coding  categories  and  development  of  coding  protocols,  code 
book,  and  data  collection  sheets;  (3)  familiarization  of  multiple 
raters  with  the  data  collection  sheets,  coding  protocol,  and  use 
of  the  code  book;  (4)  execution  of  the  coding  protocols  by  the 
trained  raters;  (5)  calculation  of  interrater  reliability  scores  and 
comparison  of  the  average  scores  to  the  70%  criterion  (Stemler, 
2004);  (6)  creation  of  actionable  feedback  categories  by  par¬ 
titioning  coded  responses  based  on  common  content/ themes 
identified  during  phase  2;  (7)  calculation  of  category  frequencies 
and  presentation  of  detailed  results. 

2. 1  Data  Review  and  Preparation 

The  first  phase  of  the  content  analysis  involved  sifting  through 
the  open  text  responses  to  the  1 1  items.  This  was  done  for  purposes 
of  (a)  becoming  familiar  with  the  various  classes  or  themes  present 
to  inform  the  coding  protocol,  and  (b)  identifying  codable  units^  to 
ensure  all  raters  coded  the  same  text. 

A  recommendation  was  defined  as  a  response  that  either  stated 
a  needed  improvement  outright  or  stated  an  issue  or  problem  that 
needs  resolution  (or  both).  There  was  no  limit  on  the  number  of 
recommendations  from  a  single  AME. 

Response  content  that  was  irrelevant  to  the  service  area  it  was 
entered  under  was  partitioned  for  later  review  by  a  naive  rater  tasked 
with  conducting  the  V&V,  prior  to  distributing  the  dataset  to  the  raters. 

2.2  Development  of  the  Coding  Protocol 

A  coding  protocol  was  developed  to  capture  the  breadth  of  content 
provided  by  AMEs.  The  coding  protocol  was  provided  to  the  raters 
as  part  of  a  code  book,  which  contained  tabled  classification  codes, 
along  with  example  wording  for  each  of  the  service  areas  and  “other 
recommendations.” 

2.3  Raters  and  Interrater  Reliability 

Three  raters  were  used  to  conduct  the  content  analysis.  The  lead 
rater  trained  each  rater  on  how  to  use  the  coding  protocols  and  data¬ 
sheets.  The  lead  rater  coded  all  responses.  The  other  raters  independently 
coded  a  minimum  of  10%  of  the  item  responses.  A  consensus-based 
interrater  reliability  estimate,  calculated  as  percent  of  times  raters  are 
in  agreement,  was  used  to  assess  consistency  in  applying  the  coding 
protocol  (Stemler,  2004;  see  also  Graham,  Milanowski,  &  Miller, 
2012).  Interrater  reliability  estimates  were  computed  and  averaged  for 


^  A  codable  unit  refers  to  a  single  distinct  improvement/ issue  that  is  meaningful 
to  the  survey  question.  Of  note,  a  single  sentence  may  contain  more  than  one 
codable  unit,  and  multiple  sentences  may  constitute  a  single  codable  unit. 


all  pairs  of  raters  across  the  1 1  open  text  items.^  The  average  overall 
reliability  score  of  83%  surpassed  the  established  criterion  of  70%, 
which  was  required  to  move  to  the  next  phase  of  the  content  analysis. 

2.4  Actionable  Feedback 

Common  content/ themes  of  all  coded  recommendations  served 
as  the  basis  for  the  actionable  feedback  categories.  Descriptions  of 
the  actionable  feedback  associated  with  each  high-level  category 
are  presented  in  the  next  section.  In  the  process  of  partitioning  the 
recommendations,  three  high-level  categories  emerged  that  framed 
results  in  programmatic  terms:  (a)  development  of  organizational 
capabilities,  (b)  enhancements  to  systems/tools,  and  (c)  changes  to 
processes  and  policies. 

3.0  RESULTS 

Results  are  reported  by  type  of  AME,  not  for  purposes  of 
comparison,  but  due  to  different  operational  conditions.  Two 
AME  groups  were  created— a  domestic  group  and  a  combined 
military,  federal,  and  international  (MFI)  group.  Descriptive 
statistics^  used  in  the  analysis  included:  frequency  (count),  percent 
(as  valid  percent,  based  on  actual  number  of  responses  to  a  survey 
item),  range  (minimum  to  maximum  value),  average  [mean  and 
median  (med)  (50^^  percentile)],  and  standard  deviation  (sd). 

Of  the  3,388  AMEs  invited  to  participate,  2,199  submit¬ 
ted  a  survey,  and  2,118  met  the  selection  criteria^  for  inclusion 
of  their  data.^  The  vast  majority  of  AME  respondents  (98.2%) 
submitted  their  responses  online  either  from  a  computer  or 
mobile  device.  The  remaining  1 .8%  returned  a  paper  version  of 
the  survey,  which  they  had  requested  and  received  in  the  mail. 

3. 1  Respondents  and  Open  Text  Response  Totals 

Approximately  one  in  five  (438  of  2, 1 1 8)  of  the  AMEs  pro¬ 
vided  a  response  to  the  final  text-entry  item  asking  for  “additional 
recommendations,  beyond  those  you  have  already  provided,  on 
how  the  FAA  can  improve  airman  medical  certification  support 
and  services.”  Of  the  1,840  domestic  AME  respondents,  392 
(21.3%)  provided  at  least  one  response.  As  for  the  278  MFI 
AME  respondents,  46  (16.5%)  provided  at  least  one  response. 

^  Raters  were  provided  refined  definitions  of  the  coding  categories  in  conjunction 
with  follow-on  training  before  being  tasked  with  independently  coding  an 
additional  1 0%  of  responses  for  those  service  areas  with  consensus  estimates 
below  criterion. 

5  Inferential  statistics  conducted  for  exploratory  purposes  are  not  reported. 

^  Selection  criteria  were  a  minimum  of  1  year  as  an  AME  and  in  the  previous 
12  months  had  examined  at  least  1  airman  applicant. 

^  See  Civil  Aerospace  Medical  Institute  (2012a,  2012b). 
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Table  3.  Counts  for  MFI  respondents,  and  coded  responses  and  comments 


AMED 

AME 

Training 

AMCD 

RFS 

Office 

Deferral 

Process 

Standards 

& 

Guidelines 

ECG 

System 

AMCS 

Med- 

XPress 

Online 
Info  & 
Pubs 

Other 

Areas 

Respondent 

3 

12 

5 

5 

7 

3 

9 

18 

11 

5 

10 

Coded 

Response 

4 

19 

4 

8 

8 

1 

9 

18 

3 

5 

22 

Coded 

Comment 

0 

5 

1 

1 

0 

0 

1 

5 

4 

0 

0 

Ollier  AME  AMCS  RFS  Oflice  EGG  0AM  AMCD  Deferral  Med-  Slarrdards  AIMED 
Training  System  Website  Process  XPress  & 

Guidelines 


Figure  1 .  Overview  of  coded  responses  across  service  areas  and  “other” 


Responses  that  met  the  definition  of  a  recommendation— /s: 
needed  improvement  stated  outright  or  an  issue/problem  that  needs 
addressing'-y^^ro,  deemed  codable.  In  total,  895  recommenda¬ 
tions  were  received.  Domestic  AMEs  provided  794  codable 
responses,  and  the  MFI  group  provided  101.  One  hundred 
and  ninety-seven  responses  did  not  meet  the  definition  and 
were  analyzed  separately  as  comments.  Tables  2  and  3  present 
breakouts  by  AME  group  of  respondents,  and  coded  responses 
and  comments  across  the  1 1  areas.  Coded  means  the  content  of 
the  text  represented  a  single  recommendation  or  comment.  A 
single  respondent  could  provide  multiple  recommendations/ 
comments  for  any  of  the  service  areas  and  “Other,”  which  was 
comprised  of  recommended  changes  to  medical  certification 
processes  and  policies. 

The  distribution  of  coded  responses  in  Figure  1  shows  the 
origin  of  the  data  that  fed  the  final  phases  of  the  content  analy¬ 
sis.  The  subset  of  recommendations  used  in  the  programmatic 
analysis  is  described  in  subsection  3.3. 

Characteristic  of  the  AMEs  that  provided  recommendations 
are  presented  next.  Although  all  responses  that  met  the  defini¬ 
tion  of  a  recommendation  were  treated  as  valid  perceptions,  the 
demographics  of  the  respondents  are  important  for  interpreting 
and  prioritizing  feedback. 


3.2  Profile  of  the  AMEs  That  Provided  an  Open  Text  Response 

The  qualifications  of  those  providing  recommendations  are, 
in  part,  reflected  in  years  of  experience  as  an  AME,  number  of 
airmen  examined  in  a  12-month  period,  and  estimated  propor¬ 
tion  of  practice  dedicated  to  aviation  medicine.  A  gauge  of  the 
representativeness  of  AMEs  providing  recommendations  can 
be  found  in  relative  comparisons  of  the  proportions  of  the  438 
respondents  providing  recommendations  to  those  who  did  not 
and  to  all  AMEs  on  the  survey  distribution  list.^ 

3.2.1  Qualifications 

Demographics  for  the  two  AME  groups  are  presented 
together  in  Figure  2  but  not  for  comparative  purposes.  Rather, 
they  are  presented  together  to  provide  a  frame  of  reference  for 
the  experiences  that  formed  the  basis  of  their  recommendations. 
Descriptive  statistics  for  some  of  the  numeric-entry  items  are 
presented  in  Table  4. 


^The  distribution  list  was  obtained  from  the  AME  Information  System  (AMEIS). 
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EH  Domestic  AMEs  H  MFI  group 


Yes 


Senior  AME 

Dorn  (n=390) 
MFI  fn=46^ 


5  or  less  6-15  16-25  26-35 


Years  as  AME 

Dorn  (n=389) 
MFI  fn=46^ 


Oor  less|51to  250i  251  to 
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751  to 
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Number  of  Exams  in  past  12  mos. 

Dorn  (n=392) 

MFI  fn=46^ 


100 

80 


^  60 

<D 

I  40 


20 

0 
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Aerospace 


Primary 

other 

Private 

Clinic 

Hospital 
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Care 

Office 

Type  of  Practice 

Dorn  (n=388) 
MFI  fn=45) 


Location  of  Practice 
Dorn  (n=391) 

MFI  fn=45) 


Figure  2.  Demographics  of  AMEs  providing  a  response 


Table  4.  Descriptive  statistics  for  demographic  numeric  entry  items 


Domestic  AMEs 

mean 

sd 

range 

median 

Years  as  AME 
(n=389) 

22.4 

13.2 

1-60 

20 

Number  of  airman 
exams  conducted 
in  12-month  period 

221 

391.1 

3-3800 

78 

Percent  of  practice 
AME-related 

25 

34.3 

0-100 

8 

MFI  group 

mean 

sd 

range 

median 

Years  as  AME 

14.6 

10.0 

1-53 

15 

Number  of  airman 
exams  conducted 
in  12-month  period 

56 

58.9 

1-300 

30 

Percent  of  practice 
AME-related 

43 

35.7 

0-100 

30 
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Figure  3.  Comparison  of  the  proportions  (with  count)  of  AMEs  that  did  and  did  not  provide  an  open  text 
response  to  their  respective  populations 


On  average,  the  domestic  group  tended  to  have  more  years 
of  experience  as  an  AME  and  have  examined  more  airman  ap¬ 
plicants  in  the  12-month  period  than  the  MFI  group.  However, 
more  of  the  MFI  group  characterized  their  practices  as  aerospace 
medicine,  while  more  of  the  domestic  group  characterized  theirs 
as  primary  care.  Patterns  were  somewhat  similar  for  their  practice 
locations — more  reported  working  out  of  private  offices  than 
elsewhere. 

3.2.2  Representativeness 

More  senior  AMEs  than  AMEs  in  general  provided  recom¬ 
mendations  on  improving  services,  which  would  be  expected 
given  the  higher  rate  of  senior  AMEs  in  the  AME  population 


(Fig  3).  However,  there  was  a  disproportionate  number  of  senior 
AME  to  AME  that  provided  recommendations  relative  to  the 
group  population.^ 

With  that  in  mind,  response  patterns  indicated  more  do¬ 
mestic  senior  AMEs  tended  to  provide  recommendations  than 
not.  Senior  AMEs  from  the  MFI  group  did  not  show  the  same 
tendency.  With  more  senior  AMEs  providing  recommendations 
for  improvements,  the  issues  they  raised  may  not  fully  reflect 
issues  challenging  AMEs  in  general;  however,  the  improvements 
and  changes  they  proposed  would  benefit  all  AMEs  at  some 
point  in  their  tenure  as  designees. 


^The  AMEIS  dataset  used  in  the  survey  effort  was  downloaded  March  2012  and 
the  survey  closed  3 1  May.  The  data’s  shelf-life  may  account  for  discrepancies 
between  the  population  and  survey  data,  since  AMEIS  is  updated  as  new 
information  becomes  available. 
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The  regional  distribution  of  the  domestic  AMEs  providing 
an  open  text  response  was  fairly  representative  of  all  domestic 
AMEs  (Fig  4).  Note  that  the  Southwest  and  Great  Lakes  regions 
were  slightly  under-represented,  and  the  Western  Pacific  region 
was  over-represented. 


Although  the  MFI  regional  distribution  is  not  delineated 
to  maintain  respondents  anonymity,  respondents  represented 
all  but  four  regions. 


Alaskan 

New  England 

Central 

Northwest  Mountain 

Southwest 

Western -Pacific 

Eastern 

Great  Lakes 

Southern 


Percent 


Figure  4.  Regional  distribution  of  domestic  respondents  relative  to  their 
population 
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3.2.3  Satisfaction  With  Aerospace  Medical  Certification 
Services 

An  examination  of  AME  satisfaction  with  certification  service 
providers  and  tools  they  had  used  in  the  past  12  months  and 
agreement  with  standards  and  guidelines  for  deferral  was  made 
between  those  who  provided  at  least  one  open  text  response 
and  those  who  chose  not  to  respond  (Table  5).  Overall,  slightly 
greater  proportions  of  domestic  AMEs  responding  to  an  open 
text  item  tended  to  report  not  being  satisfied,  compared  to  those 
choosing  not  to  respond.  The  reverse  was  found  with  the  MFI 
group  for  half  the  service  areas  rated. 


Table  5.  Comparison  of  AMEs  NOT  satisfied*  with  airman  medical  certification  services  that  did 
and  did  not  provide  an  open  text  response 


Domestic  AMEs 

response 

no  response 

%  difference'^ 

% 

n 

% 

n 

AMED 

15.2 

329 

10.6 

1485 

5.4 

AMCD 

16.5 

315 

9.3 

1089 

7.2 

RFS 

15.7 

324 

11.0 

1157 

4.7 

AMCS 

16.9 

388 

10.3 

1277 

6.6 

MedXPress 

20.6 

340 

15.0 

1176 

5.6 

ECG  System 

27.2 

250 

21.6 

745 

5.6 

0AM  Website 

21.7 

166 

15.2 

426 

6.5 

Standards  &  Guidelines  for 
deferrals  are  medically 
reasonable  and  appropriate 

26.3 

388 

19.3 

1385 

7.0 

MFI  Group 

response 

no  response 

%  difference'^ 

% 

n 

% 

n 

AMED 

13.2 

38 

13.9 

187 

-0.7 

AMCD 

19.4 

31 

18.4 

136 

-1.0 

RFS 

33.3 

18 

16.7 

72 

16.6 

AMCS 

20.5 

44 

19.3 

223 

1.2 

MedXPress 

17.2 

29 

10.7 

140 

6.5 

ECG  System 

5.3 

1 

5.6 

18 

-0.3 

0AM  Website 

5.0 

20 

13.0 

69 

-8.0 

Standards  &  Guidelines  for 
deferrals  are  medically 
reasonable  and  appropriate 

13.1 

38 

10.2 

196 

2.9 

*NOT  satisfied  was  calculated  as  the  sum  of  the  percentages  of  AMEs  selecting  Neither,  Dissatisfied,  and 

Very  dissatisfied. 

^%Difference  was  calculated  as  %response  minus  %no  response. 
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Table  6.  Comparison  of  reported  use  of  services*  in  the  past  12  months  by  AMEs  (count)  that  did  and 
did  not  provide  an  open  text  response 


Domestic  AMEs  (n=392) 

AMED 

AME 

Training 

AMCD 

RFS 

Office 

EGG 

System 

AMCS 

Med- 

XPress 

0AM 

Website 

Used 

15 

91 

34 

61 

41 

58 

93 

35 

Did  not 

1 

0 

5 

8 

12 

4 

17 

4 

MFI  Group  (n=46) 

AMED 

AME 

Training 

AMCD 

RFS 

Office 

EGG 

System 

AMCS 

Med- 

XPress 

0AM 

Website 

Used 

3 

12 

5 

5 

1 

18 

8 

4 

Did  not 

0 

0 

0 

0 

8 

0 

3 

1 

*No  comparable  survey  items  regarding  use  of  deferrals  or  standards  &  guidelines. 
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Figure  5.  Distribution  of  AME  recommendations  across  the  high-level  categories 


Satisfaction  ratings  were  collected  only  if  the  respondent 
reported  use  of  the  service  within  the  past  12  months.  There 
was  no  such  restriction  on  the  open  text  items.  Shown  in  Table 
6  are  the  numbers  of  AMEs  providing  an  open  text  response  to 
a  particular  service  area  based  on  whether  or  not  the  service  had 
been  used  in  the  past  12  months. 

3.3  Analysis  of  Recommended  Improvements  in  Actionable 
Terms 

Ninety-six  percent  (861  of  895)^^  of  the  coded  responses 
were  used  in  the  final  phases  of  the  content  analysis.  Of  those, 
80%  (691)  were  directly  related  to  the  high-level  categories.  The 
remaining  170  recommendations  were  indirectly  related  to  the 
higher  categories  and  pertained  to  AME  training  (94),  Standards 
&  Guidelines  (52),  class  of  certification  requirements  (18),  and 
deferral  tracking  (6). 


Figure  5  shows  a  similar  pattern  between  groups  in  the  dis¬ 
tribution  of  recommendations  across  the  high-level  categories. 

Results  are  presented  in  actionable  terms  for  each  of  the 
high-level  categories  in  the  following  sections:  3.3. 1  Development 
of  organizational  capability ,  3.3.2  Enhancements  to  digital  sy stems! 
tools,  and  3.3.3  Changes  to  process  and  policy. 


^^Recommendations  excluded  from  the  in-depth  analysis  were  those  coded 
as  “miscellaneous.” 


8 


Table  7.  Distribution  (percent)  of  recommendations  across  service  providers 


OAM 

AMCD 

RFS 

AMED 

Domestic  AME 

(290  recommendations) 

16.6 

42.6 

22.5 

18.3 

MFl  group 

(40  recommendations) 

15.0 

40.0 

20.0 

25.0 

Table  8.  Actionable  feedback  categories  for  developing  organizational  capability 


Communication 

(timely,  content) 

-  timeliness  of  returned  phone  calls  from  FAA  physicians 

-  content  of  communication 

Cost 

-  reduce  travel  expenses  for  training 

-  reduce  cost  of  equipment/transmission  requirements 

FAA  accessibility 

(real  time/anytime) 

-  contact  with  FAA  physician  while  airman  in  office 

-  access  to  FAA  physician  during  off  hours 

FAA  decision 

-  streamline  process  to  shorten  decision  time 

-  consistency 

FAA  personnel 

(interactions,  number) 

-  FAA  personnel  need  to  be  of  service 

-  treat  AME  professionally  with  courtesy/respect 

-  more  staff 

Information  request 

-  FAA  personnel  contact  information 

-  online  access  to  specific  content 

-  notification  of  updates,  publications,  retraining 

Training  need 

-  lack  sufficient  coverage  of  content  in  AME  training 

-  gaps  in  airman  education 

-  qualifications  lacking  in  FAA  personnel 

3.3. 1  Development  of  Organizational  Capability 

The  analysis  examined  a  subset  of  330  recommendations^^ 
compiled  from  the  following  sources:  AMED  (includes  AME 
training),  RFS  offices,  AMCD  (includes  AMCS  and  the  deferral 
process),  and  OAM  (includes  standards  &  guidelines,  MedXPress, 
and  online  information  and  publications).  The  pattern  of  results 
shown  in  Table  7  likely  reflects  the  frequency  of  AME  contact 
across  the  service  providers,  with  AMCD  having  the  most  contact 
with  the  respondents.  The  percentage  of  recommendations  for 
the  service  providers  was  relatively  similar  between  the  groups, 
with  the  exception  of  the  AMED.  The  distinct  difference  in  the 
percentages  of  recommendations  for  AMED  is  due  to  the  MFl 
group  proportionally  identifying  more  AME  training  needs  than 
the  domestic  AMEs. 


The  actionable  feedback  categories,  defined  in  Table  8, 
are  pertinent  to  AMEs  effectively  and  efficiently  performing 
their  responsibilities  as  FAA  designees.  The  categories  are  the 
common  themes  that  emerged  during  the  coding  phase  of  the 
content  analysis. 


"Asa  reminder,  a  valid  recommendation  either  stated  how  to  improve  current 
services  or  identified  a  current  problem  with  adequacy  of,  consistency  in,  and 
delivery  of  services. 
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□  Domestic  AMEs  (290  recommendations)  ■  MFI  group  (40  recommendations) 


training  need  FAA  accessibility  information  communication  FAA  decision 
(realtime/  request  (timelVi  content) 

anytime) 

Figure  6.  Distribution  of  feedback  for  development  of  organizational  capability 


cost 


FAA  personnel 
(interaction^ 
number) 


Table  9.  Training  needs  (count)  from  domestic  AMEs 


100% 


sz 

■S  60% 


40% 


20% 


0% 


AMEs,  74% 


Airmen, 

13% 


FAA 

personnel, 

13% 


Audience 

(55) 

Topic 

AMEs* 

(41) 

-  certification  issues/cases  (13) 

-  MedXPress  (11) 

-  updates/changes  (4) 

-  specialty  (3)  [ophthalmology  (1), 
cardiology  (1),  space  medicine  (1)] 

-  condition  (2)  [obesity  (1),  sleep 
apnea  (1)] 

-  ECG  transmission  (2) 

-  administrative/AME  staff  (2) 

-  Mike  Monroney  Aeronautical 

Center  (MMAC)  tour  (1) 

Airmen 

(7) 

-  MedXPress  (5) 

-  role  of  FAA  and  AME  (1) 

-  more  education  (1) 

FAA 

personnel 

(7) 

-  tech  support  (3)  [MedXPress,  ECG, 
RFS] 

-  AMCD  staff  (3)  [hotline  (1),  ECG 
filing  (2)] 

-  RFS  staff  (1) 

*Three  recommendations  pertained  to  updating  content  in  the  seminars  and  online. 


The  distribution  of  recommendations  across  the  categories 
(Fig  6)  is  sorted  highest  to  lowest  cumulative  percent.  Detailed 
results  follow  in  that  order. 

The  next  phase  of  the  analysis  involved  assigning  secondary 
categories  and  computing  their  frequencies.  Detailed  results  for 
each  feedback  category  are  presented  next  in  the  tabled  format 
described  above  by  group  when  they  offered  five  or  more  rec¬ 
ommendations.  With  the  exception  of  the  “training  need”  and 
“costs”  categories,  the  results  are  presented  by  agency  based  on 
origin  of  the  response. 


Training  Needs.  Specified  training  topics  for  AMEs,  identi¬ 
fied  need  for  airman  education,  and  deficits  in  FAA  personnel 
performance  are  found  in  Tables  9  and  10.  Most  recommenda¬ 
tions  from  domestic  AMEs  indicated  an  interest  in  furthering 
their  understanding  of  certification  requirements  for  the  range 
of  medical  issues  they  may  face.  Half  (7  of  1 3)  of  their  requests 
for  more  training  on  certification  issues  indicated  a  preference 
for  use  of  case  studies.  The  domestic  group  also  indicated  a  gap 
in  their  and  airmens  understanding  and  use  of  MedXPress,  with 
an  emphasis  on  addressing  it  in  the  near-term.  The  MFI  group 
referred  to  the  same  training  needs,  in  addition  to  training  on 
AMCS  use  and  ECG  transmission. 
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Table  10.  Training  needs  (count)  from  the  MFI  group 


100%  ^ 

Audience 

(13) 

Topic 

-  certification  issues  (4)  [deferrals  (2), 

80%  - 

complex  cases  (2)] 

n 

AMEs 

-  MedXPress  (2) 

■jM 

(U 

(11) 

-  AMSC  (2) 

c  60%  - 

c 

AMEs,  85% 

-  ECG  transmission  (2) 

t 

E 

o 

-  aeromedical  evacuation  (1) 

o  ■ 

Airmen 

-  MedXPress  (1) 

20% 

(2) 

-  more  education  (1) 

Airmen, 

15% 

0%  - 

AME  Training.  AMEs  identified  needed  improvements  to 
their  training  beyond  the  specific  topics  reported  above.  Both 
groups  provided  training  recommendations  on  frequency  of 
offerings,  strategies  (methods  and  media),  locations,  and  ways 
to  add  value  that  would  offset  the  cost.  Table  1 1  presents  results 
for  both  AME  groups. 
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Table  1 1 .  Distribution  of  additional  recommended  improvennents  (count=94)  to  AME  training 
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AsMA=Aerospace  Medical  Association;  CME=continuing  medical  education;  HIMS=Human  Intervention  Motivation  Study; 
NASA=National  Aeronautics  and  Space  Administration;  Q&A=question  and  answer;  UK=United  Kingdom 


FAA  Accessibility  (real  time! anytime).  A  recurrent  needed 
improvement  is  to  provide  more  access  to  FAA  physicians  at  the 
RFS  offices  and  AMCD.  The  need  stems  from  questions  that 
arise  during  the  applicant  s  appointment  that  negatively  impacts 
effectiveness  and  efficiency  of  the  certification  procedure  if  the 
AME  is  unable  to  contact  the  FAA.  A  related  challenge  for  AMEs 


is  the  need  for  information  that  only  an  FAA  physician  can  provide 
when  the  FAA  offices  are  closed.  Several  solutions  to  anytime 
access  were  proposed— specifics  are  presented  in  Tables  12  and 
13.  (Note  that  responsiveness  of  FAA  personnel  to  AME  voice- 
and  e-mail  is  a  component  of  the  “communications”  category.) 


Table  12.  FAA  accessibility  issues/solutions  (count)  from  domestic  AMEs 


100% 


80% 


«  60% 


O  40% 


20% 


0% 


real  time 
contact, 
30% 


easy/tast 
access,  19% 


extended 
hours,  18% 


more 
phone 
lines,  16% 


anytime 
access,  10% 


online 
contact  0% 


Issue/Solution  (67) 

AMCD 

(35) 

-  extended  hours  (8)  [to  Pacific  Time 
Zone  (4),  after-hours  access  (3),  for 
tech  support  (1)] 

-  easy/fast  access  (7)  [to  physician 
(5),  to  tech  support  (1),  by  phone  (1)] 

-  more  phone  lines  (7)  [add  toll-free 
(2),  direct  to  division  chief  (1),  direct 
to  physician  on  case  (1),  for  AMCS 
tech  support  (1),  add  voice  mail  (1), 
add  lines  (1)] 

-  real  time  contact  (6)  [with  physician 
(5),  to  verify  issuance  (1 )] 

-  anytime  access  (4)  [on-call  (2),  24/7 
AMCS  tech  support  (2)] 

-  online  contact  (3)  [for  questions  (2), 
offer  chat  (1)] 

RFS 

(31) 

-  real  time  contact  (14)  [with  person 
(10),  with  physician  (4)] 

-easy/fast  access  (5)  [to  office  (4),  to 
physician  (1)] 

-extend  hours  (4) 

-  more  phone  lines  (4)  [direct  to 
physician  (2),  add  toll-free  (2)] 

-  anytime  access:  on-call  (3) 

-  Online  contact  (1) 

AMED 

(1) 

-  easy/fast  access  (1) 

Table  13.  FAA  accessibility  issues/solutions  (count)  from  the  MFI  group 


100% 


80% 


c 

o 

-D  60% 


E 

E 

o 

u 

<u 

^  40% 


20% 


0%. 


online 

contact, 

33% 

real  time 

contact, 

22% 

easy /fast 
access,  11% 

extended 
hours,  11% 

anytime 
access,  11% 

more 

phone 
linp^  11% 

issue/Solution  (9) 

AMCD 

(4) 

-  online  contact  (1) 

-  real  time  contact  (1) 

-  easy/fast  access  (1) 

-  anytime  access  (1) 

RFS 

(4) 

-  online  contact  (2) 

-  extended  hours  (1) 

-  more  phone  lines  (1):  direct  to  RFS 

AMED 

(1) 

-  online  contact  (1) 

13 


The  groups  raised  similar  issues — to  be  effective  and  efficient, 
they  rely  on  guidance  and  decision  making  from  FAA  physicians 
and  technical  support  for  AMCS,  in  particular.  At  hand  is  that 
the  need  for  assistance  from  AMCD  is  often  during  an  exam  or 
outside  of  FAA  office  hours,  especially  for  AMEs  in  the  Pacific 
Time  Zone  and  abroad.  For  the  domestic  AMEs  in  six  of  the 
nine  regions,  difficulty  making  contact  with  RFS  office  staff  was 
most  frequently  noted. 

Information  Request.  AMEs  expressed  a  need  for  informa¬ 
tion  to  help  them  do  their  jobs.  The  specifics  are  in  Tables  14  and 
15.  The  domestic  AMEs  had  more  variable  information  needs 
than  the  MFI  group.  The  domestic  group  expressed  a  need  for 
more  reference  materials  as  decision  aids  and  material  to  assist 
airmen  in  meeting  special  issuance  (SI)  requirements,  whereas, 
some  from  both  groups  expressed  interest  in  information  on 
ECG  equipment. 


The  manner  of  distributing  the  requested  information  was 
not  always  specified;  however,  when  it  was  noted,  there  was  a 
tendency  toward  online  access  (11  of  17).  If  the  information 
they  need  is  currently  available,  then  the  request  reveals  that 
critical  information  is  not  reaching  some  AMEs,  and  further 
investigation  may  be  warranted. 

Communication  (timelinessi content).  A  recurrent  needed 
improvement  at  AMCD  and  the  RFS  offices  (9  of  the  1 1  are 
in  the  Southern  region)  was  to  be  more  responsive  to  AMEs; 
and  for  AMCD  is  to  be  more  communicative.  The  MFI  group  s 
four  recommendations  pertained  solely  to  AMCD — with  three 
requests  to  be  responsive  to  AMEs  and  one  request  to  inform 


Table  14.  Information  requests  (count)  from  domestic  AMEs 


inrifli 

Type  of  information  (52) 

S0%  - 

reference 

material, 

33% 

0AM 

(29) 

-  reference  material  (15)  [list  of 
acceptable/unacceptable  medications 
(7),  conditions  and  requirements  (4), 

AME  physicians  desk  reference(l), 
quick  reference  (1),  FAA  standards  (1), 
case  studies  (1)] 

-  website  content  (5)  [frequently  asked 
question  (FAQ)  (1),  FAQ  for  MedXPress 
(1),  renal  calculi  (1),  updates  (1),  my 
contact  info  (1)] 

60%  ^ 
Vi 

c 

o 

4-J 

AME  Guide, 

13% 

-  for  airmen  (3)  [template  for  treating 
physician  (1),  SI  requirements  handout 
(1),  electronic  medical  release  form  (1)] 

-  AME  Guide  hardcopy  (3)  or  CD  (1) 

-  publication  (2)  [aviator  stressors  in 

FASB  (1),  reoccurring  certification 
problems  and  solutions  (1)] 

"D 

C 

E 

£ 

ECG 

system. 

o 

u 

cc: 

13% 

-  ECG  system  (7)  [equipment  options  (4), 
transmission  options  (3)] 

-  contact  info  (2):  phone  list 

U  40%  - 

website 

content. 

AMCD 

(13) 

10% 

-  reference  material  (2)  [exam  room 

contact 

info,  8% 

poster,  conditions  and  requirements] 

-  publication  (1):  annual  SI  report 

20%  - 

training 

-  AME  Guide  revised  by  clinician  (1) 

content, 

8% 

RFS 

-  RFS  function  and  responsibilities  (2) 

publication, 

6% 

(3) 

-  contact  info  (1):  Southwest  phone  list 

-  training  content  (4)  [searchable  (2), 
mailed  (1),  equipment  list  (1)] 

rnaLeiial  for 
airman,  6% 

AMED 

(7) 

0%  - 

function  of 

RF5.4% 

-  AME  Guide  hardcopy  (3)  [provided 
annually,  for  fee,  at  seminars] 
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airmen  regarding  deferral  timeframe.  The  domestic  group  s  com¬ 
munication  recommendations  are  shown  in  Table  16.  Beyond 
their  specified  need  for  notification  when  certification  policies 
change,  confirmation  of  receipt  of  ECG  transmissions,  and  in¬ 
formation  on  training  due  dates;  to  improve  their  performance. 


several  domestic  AMEs  asked  for  corrective  feedback  on  their 
certification  decisions  and  deferrals  sent  to  AMCD.  Some  also 
asked  that  FAA  communications  with  them  and  the  airmen 
contain  full  details  of  requirements  and  decision  timelines. 


Table  15.  Information  requests  (count)  from  the  MFI  group 


100% 


80% 


60% 

b 

F 

o 

u 

iD 

o  ^0% 


20% 


0% 


contact 

info,  38% 

website 

content, 

25% 

ECG 

system, 

25% 

reference 

material, 

13% 

Type  of  information  (8) 

OAM 

(2) 

-  website  content  (2)  [list  of  all 
info/pubs  (1),  more  on  oncology  (1)] 

AMCD 

(6) 

-  contact  info  (3)  [email  addresses 
(2),  point  of  contact  (1 )] 

-  ECG  system:  equipment  options 
(2) 

-  reference  material:  conditions  and 
requirements  (1) 

Table  16.  Communication  issues/solutions  (count)  from  domestic  AMEs 


100%  H 


80% 


60% 


E 

E 

o 

u 

qy 

££ 

O  ^% 


20% 


0%  J 


notSf I  cation^ 
4G% 


be 

responsive, 

28% 


rnntpnit^ 

18% 


with 

airmen,  B% 


Issue/Solution  (50) 

OAM 

(3) 

-  to  airmen  (2)  [guidelines  on  SI 
requirements:  for  selective  serotonin 
reuptake  inhibitors  (SSRIs)  (1),  for 
letter  from  treating  physician  (1)] 

-  notification  (1):  via  email  of  new 
publication 

AMCD 

(32) 

-  notification  (19)  [changes  in  standards 
&  guidelines  (10),  ECG  receipt 
confirmation  (7),  when  certificate  is 
issued  (1),  via  email  (1)] 

-  content  (7)  [feedback  on  deferral 
decisions  (4),  complete  requirements 
(1),  decision  timeframe  (1),  include  DoB 
and  exam  date  when  referencing  an 
applicant  (1)] 

-  be  responsive  (4) 

-  to  airmen  (2):  fully  inform  them 
regarding  deferral  timeframe  etc 

RFS 

(11) 

-  be  responsive  (9) 

-  content  (2):  feedback  corrective  action 

AMED 

(4) 

-  notification  (3):  when  retraining  is  due 

-  be  responsive  (1) 
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FAA  Decision.  The  three  issues  raised  by  the  AME  groups 
pertained  to  AMCD  and  their  RES  office:  (a)  lack  of  consistency 
in  decisions  (across  regions  and  FAA  physicians),  (b)  too  lengthy 
of  a  decision  process  for  deferrals,  and  (c)  need  for  decisiveness 
when  calling  for  a  decision  or  guidance.  The  MFI  group  s  three 
recommendations  were:  two  requests  for  faster  turnaround  from 
AMCD  and  an  expressed  need  for  decisiveness  from  the  RES 
office.  The  majority  (18  of  20)  of  issues/solutions  raised  by  the 
domestic  group  focused  on  AMCD.  Specifics  are  presented  in 
Table  17. 


Cost.  The  two  main  issues  raised  by  the  AME  groups  were 
costs  associated  with  time  and  travel  for  training  and  ECG 
equipment  and  transmission  costs.  The  MFI  group  noted  three 
issues/solutions:  the  costliness  of  ECG  equipment  and  the  need 
to  reduce  and/or  cover  training  travel  costs.  Specific  issues/solu- 
tions  from  the  domestic  group,  presented  in  Table  18,  are  nearly 
evenly  split  between  the  ECG  system  and  training. 


Table  17.  FAA  decision  issues/solutions  (count)  from  domestic  AM Es 


Issue/Solution  (20) 

AMCD 

18) 

-  speedier  (13)  [faster  (7),  streamline 
the  process  (4),  no  more  than  4 
weeks  (3)] 

-  consistent  (5)  [across  physicians 
(2),  regions  (1),  predictable  (1)] 

RFS 

2) 

-  consistent  (1):  across  regions  and 
with  AMCD 

-  decisive  (1) 

100% 


80% 


-g  60% 
c 

cj 

E 

E 

o 

u 

^  40% 
O 


20% 


0% 


speedier, 

65% 


consistent, 

30% 


decisive, 

— m — 


Table  18.  Cost  issues/solutions  (count)  from  domestic  AMEs 


100% 


80% 


60% 


40% 


20% 


0%  -• 


ECG 

system, 

5S% 

training, 

42% 

issue/Soiution  (19) 

ECG 

System 

(11) 

-  expensive  to  transmit  (5)  [change 
mode  to  save  costs  (1),  cheaper 
ways  to  transmit  (1),  requires 
dedicated  phone  line  (1),  add  toll-free 
line  (1),  transmission  (1)] 

-  costly  equipment  (4)  [expensive 
machine  (2),  use  less  expensive 
equipment  (1 ),  not  cost  effective  to 
replace  (1)] 

-  expensive  overali  (2)  [very 
expensive  (1),  unnecessary  costs  (1)] 

Training 

(8) 

-  too  expensive  (8)  [cannot  justify  the 
cost  of^me  to  travel  (6),  very 
expensive  (1 ),  skip  costly  7-year  fly-in 
(1)] 
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FAA  Personnel.  Issues/solutions  raised  by  domestic  AMEs 
concerned  the  need  for  more  FAA  personnel  and  service-oriented 
support.  Specifics  are  presented  in  Table  19.  The  MFI  group  did 
not  raise  any  issues  with  FAA  personnel. 

Summary  of  Results  for  Development  of  Organizational 
Capability.  Tables  20  and  21  contain  summaries  of  the  330 
AME  recommendations  for  developing  organizational  capability. 
Table  20  provides  a  programmatic  perspective  by  showing  the 
breakout  of  feedback  within  each  category  across  the  organizations 
serving  AMEs.  Table  21  provides  the  individual  organizations 
a  perspective  on  which  capabilities  need  improving  by  showing 
the  distribution  of  recommendations  across  feedback  categories. 

Highlights  in  Table  20  indicate  the  organizational  level  where 
the  majority  of  recommended  improvements  reside  for  each 
feedback  category.  Most  of  the  feedback  pertained  to  AMCD 
(Table  21),  which  is  not  surprising,  given  they  are  more  likely  to 


have  contact  with  AMEs,  as  noted  earlier  in  the  report.  AMEs 
proposed  on-call  physicians  and  online  contact  as  possible  solu¬ 
tions  to  the  lack  of  timely  guidance  and  decision  making  from 
AMCD.  They  also  expressed  issues  with  the  responsiveness  of 
AMCD  in  providing  notification  of  changes  in  standards  and 
guidelines,  deferral  status,  and  receipt  of  supporting  documen¬ 
tation,  in  particular,  and  seemingly  slow,  inconsistent  decision 
making  process  for  deferrals,  in  general.  Expenses  associated 
with  conducting  ECGs  were  noted  more  than  the  cost  to  attend 
training,  with  changes  in  transmission  mode  and  equipment 
proposed  as  cost  reducing  solutions.  The  need  for  development 
of  FAA  personnel  was  indicated  particularly  in  the  amount  and 
quality  of  interactions  with  RFS  staff.  Information  requests  were 
mainly  for  decision  aids  and  form  templates,  a  change  that  war¬ 
rants  review  by  OAM. 


Table  19.  FAA  personnel  issues/solutions  (count)  from  domestic  AMEs 


100% 


80% 


60% 


O 


10% 


20% 


0% 


be 

respectful/ 
polite,  41% 

be  of 

service, 

2Z% 

more  face- 
to-face, 

22% 

add  staff, 

15% 

Issue/Solution  (27) 

AMCD 

(9) 

-  add  staff  (4)  [decision  makers  (2), 
AMCS  tech  support  (2)] 

-  be  respectful/polite  (3)  [courteous 
(2),  accept  suggestions  (1)] 

-  be  of  service  (2)  [ECG  support  (1 ), 
stop  stonewalling  (1)] 

RFS 

(16) 

-  more  face-to-face  time  (6)  [with 

RFS  (4),  with  RFS  office  staff  (2)] 

-  be  respectful/polite  (6) 

[pleasant/personal  (3),  courteous  (2), 
aware  of  AME’s  time  (1 )] 

-  be  of  service  (4) 

AMED 

(2) 

-  be  respectful/polite  (2) 

Table  20.  Breakout  of  each  feedback  category  across  service  providers 


(Shading  indicates  highest 

percent  in  each  feedbac 

k  category.) 

Training 

need 

FAA 

accessibility 

Information 

request 

FAA 

communication 

FAA 

decision 

Cost 

FAA 

personnel 

OAM 

12% 

- 

52% 

5% 

- 

- 

- 

AMCD 

“ 

51% 

32% 

67% 

87% 

64% 

ECG  System 

33% 

RFS 

“ 

46% 

5% 

20% 

13% 

- 

59% 

AMED 

76%  AME 

1 3%  airman 

3% 

11% 

8% 

- 

36% 

AME  training 

8% 

Total 

count* 

68 

76 

60 

54 

23 

22 

27 

*Total  count  is  the  sum  of  the  number  of  recommendations  from  both  groups;  therefore,  a  greater  contribution  from 
the  MFI  group  to  the  cumulative  percent  will  result  in  a  smaller  count  than  the  reverse. 
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Regarding  AME  training  offered  by  AMED,  beyond  the 
specific  content/subject  matter  AMEs  requested,  they  also  in¬ 
dicated  the  need  for  variable  training  locations,  more  offerings, 
and  on-demand  training.  Online  training  such  as  self-paced, 
multimedia  courses  and  webinars  would  help  reduce  training 
costs  AMEs  incur,  and,  when  done  right,  could  support  some 
of  their  recommended  instructional  methods  (e.g.,  case  studies, 
demonstration,  small  group  discussion,  and  speakers). 

3.3.2  Enhancements  to  Systems/Tools 

The  analysis  examined  a  subset  of  191  recommendations 
relevant  to  the  digital  ECG  System,  AMCS,  MedXPress,  online 
AME  Guide,  and  OAM  website — systems/tools  AMEs  rely  on 
to  perform  their  duties  as  FAA  designees.  There  were  distinct 
differences  between  the  groups  (Table  22),  which  likely  reflects 


differences  in  levels  of  experience  with  the  various  systems/tools. 
The  domestic  AMEs  recommended  improvements  across  the 
board,  while  the  MFI  group  focused  on  three  systems/ tools. 

The  actionable  feedback  categories,  defined  in  Table  23, 
are  pertinent  to  designing  digital  systems/tools  for  effective 
and  efficient  use  and  are  referred  to  as  “ilities.”  The  categories 
are  the  common  themes  for  needed  improvements  to  systems/ 
tools  that  emerged  during  the  content  analysis.  It  is  important 
to  note  that  the  ilities  are  not  mutually  interdependent.  For 
instance,  an  interface  may  be  easy  to  navigate  and  intuitive,  but 
unless  the  system/tool  effectively  supports  the  user  in  meeting 
job  requirements,  it  will  have  limited  utility.  The  same  goes  for 
a  system/tool  that  cannot  be  consistently  accessed  or  is  unstable 
once  accessed — it  has  limited  utility — unreliable  systems/ tools 
also  introduce  unnecessary  frustrations. 


Table  21 .  Distribution  of  recommendations  for  each  service  provider 


(Total  count*) 

Training 

need 

FAA 

accessibility 

Information 

request 

FAA 

communication 

FAA 

decision 

Cost 

FAA 

personnel 

AMCD 

(137) 

- 

28% 

14% 

26% 

15% 

10% 

7% 

AMED 

(83) 

72% 

2% 

8% 

5% 

“ 

10% 

2% 

RFS 

(68) 

51% 

4% 

4% 

4% 

24% 

OAM 

(42) 

19% 

- 

74% 

7% 

“ 

- 

- 

*Total  count  is  the  sum  of  the  number  of  recommendations  from  both  groups. 


Table  22.  Distribution  (percent)  of  recommendations  across  systems/tools 


AMCS 

ECG 

System 

MedXPress 

Online 

AME 

Guide 

OAM 

Website 

Domestic  AMEs 
(170  recommendations) 

34.1 

20.0 

29.4 

8.8 

7.6 

MFI  group 

(21  recommendations) 

76.2 

19.0 

— 

4.8 

— 

Table  23.  Actionable  feedback  categories  for  enhancing  systems/tools 


Add  capability 

-  add  function/feature  not  currently  supported 

Adjust  capability 

-  change  to  function  currently  supported 

-  change  to  existing  interface  feature  (e.g.,  button,  text  box,  menu) 

Reliability 

-  server/software  needs  to  be  available  and  stable 

-  transmission  failures  need  to  be  reduced  and  availability  improved 

-  input  accurately  displayed/saved 

Usability 

-  make  the  interface/printouts  easier  to  use  to  accomplish  tasks 

-  make  more  user  friendly 

Utility 

-  support  users  in  effectively/successfully  performing  required  tasks 

-  support  practical  electronic  transmission  modes 

-  address  challenges  to  end  users  (e.g.,  computer  access  and  skills) 
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The  distribution  of  recommendations  across  the  categories 
is  presented  in  Figure  7. 

The  next  phase  of  the  analysis  involved  assigning  secondary 
categories  and  computing  their  frequencies.  Detailed  results 
follow  for  the  feedback  categories,  going  from  highest  to  lowest 


cumulative  percent  (Fig  7).  As  a  reminder,  some  recommended 
improvements  are  stated  as  issues  or  problem  areas  that  need 
addressing  and  others  are  stated  as  solutions.  Results  in  this 
sections  tables  are  ordered  highest  to  lowest  frequency  from 
top  to  bottom  row. 


60 


40 


c 

o 

0. 


20 


0 


add  capability  reliability  utitlity 

Figure  7.  Distribution  of  feedback  on  enhancing  systems/tools 


usability  adjust  capability 
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Add  Capability.  The  AME  groups  identified  embedded 
guidance  in  AMCS  for  themselves  and  in  MedXPress  for  the 
airmen  (Tables  24  and  25)  as  the  main  improvement  to  the 
systems/tools.  The  embedded  guidance  would  expedite  the 
application  process  by  (a)  reducing  airmen  application  errors, 


which  take  time  for  AMEs  to  correct  and  explain;  (b)  reducing 
calls  to  the  RES  office  and  AM  CD,  with  appropriate  guidance 
to  aid  AME  decisions;  and  (c)  educating  airmen  on  needed  sup¬ 
porting  documentation  for  their  medical  conditions  and  AMEs 
on  updates  to  certification  standards  and  guidelines. 


Table  24.  “Add  capability”  issues/solutions  (count)  from  domestic  AMEs 


100% 


80% 


60% 


40% 


20% 


0% 


embedded 
guidance/link 
,  33% 


auto-fill  data, 
17% 


error 

correction, 

17% 


search 
options,  9% 


download- 

ablefiles,4-% 

- - 

recognition, 

_ 


password 
reset,  4% 


marked 
changes,  4% 


Issue/Solution  (22) 

AMCS 

(11) 

-  embedded  guidance/link  (5)  [pop¬ 
up  if  condition  does  not  meet 
standard  (3),  link  direct  to  guideline 
(1),  prompts  (1)] 

-  auto-fill  data  (2)  [date  (1),  check 
“yes-has  seen  physician”  when 
adding  physician  or  medication  (1)] 

-  error  correction  (2)  [spell  check 
(1),  until  exam  complete  (1)] 

-  reset  password  (1)  online 

-  speech  recognition  (1) 

MedXPress 

(8) 

-  embedded  guidance/link  (3)  [for 
airman:  requirements  for  supporting 
documents  (2),  flying  restrictions  (1)] 

-  search  options  (2):  application 
retrieval 

-  auto-fill  (2)  [in  general  (1),  pass 
history  (1)] 

-  error  correction  (1):  assist  airman 

0AM 

Website 

(2) 

-  embedded  guidance/link  (1)  for 

quick  access  of  info  during  exam 

-  downloadable  pdf  files  (1)  so 

readily  accessible 

Online  AME 

Guide 

(1) 

-  marked  changes  (1) 

20 


Table  25.  “Add  capability”  issues/solutions  (count)  from  the  MFI  group 


100% 


80% 


■S  60  % 


40% 


:^o% 


0%  -I 


embedded 
guidance/ 
link,  44% 


new 

button, 

22% 


view 

certificate. 

22% 


track 

changes, 

11% 


Issue/Solution  (9) 


AMCS 

(8) 

-  embedded  guidance  (4) 

[warning/link  given  airman  history 
(2),  suggestions  while  completing 
form  (1 ),  better  link  to  AME  guide 
(1)] 

-  view  certificate  (2)  [before  final  (1), 
button  (1)] 

-  add  button  (2)  [SUBMIT  (1),  tech 
support  contact  info  (1)] 

Online  AME 

Guide 

(1) 

-  Track  changes  (1):  to  print  out 
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Reliability.  AMEs  brought  attention  to  their  need  for 
systems/tools  that  are  trustworthy,  i.e.,  accessible  on-demand 
and  stable  once  accessed,  with  no  concern  for  data  integrity 
(once  entered,  not  subject  to  corruption  or  loss).  All  four  reli¬ 
ability  issues  raised  by  the  MFI  group  pertained  to  improving 


the  stability  and  data  integrity  of  AMCS.  The  domestic  group 
addressed  reliability  issues  with  AMCS,  MedXPress,  and  ECG 
transmission  (Table  26).  They  are  asking  for  alternative  means 
of  completing  data  entry  off-line  when  systems  are  unavailable 
to  eliminate  slowdowns  and  frustrations. 


Table  26.  Reliability  issues/solutions  (count)  from  domestic  AMEs 


100% 


80% 


eo% 


A0% 


20% 


0% 


ready 

access,  3S% 


bldble 

connection, 

30% 


built-in 

redundancy, 

15% 


data 

intCjgrity 

assurance, 

13% 


less  lagtime, 
5% 


Issue/Solution  (40) 

AMCS 

(22) 

-  ready  access  (7)  [not  available  (3), 
cannot  process  applications  (2), 
difficult  to  access  (2)] 

-  data  integrity  assurance  (5)  [auto¬ 
save  (2),  unintended  changes  (2), 
lost  data  (1)] 

-  stable  connection  (4)  [lost 
connection  disrupts  printing  (2), 
dropped/freezes  (2)] 

-  built-in  redundancy  (4)  [backup 
system  (3),  app  for  off-line  completion 
(1)] 

-  less  lag  time  (2)  slows  process 

MedXPress 

(11) 

-  ready  access  (6)  [less  down  time 
(3),  need  24/7  availability  (1),  ready 
for  Oct  1  (1),  airmen  (1)] 

-  built-in  redundancy  (3)  [backup 
system  (1),  alternative  (1)] 

-  stable  connection  (2)  [fragile  (1),  for 
airmen  (1)] 

ECG 

System 

(7) 

-  stable  connection  (4)  [rejects 
transmission  (2),  failure  due  to  queue 
length  (1),  more  reliable  (1)] 

-  ready  access  (3)  [need  24/7 
availability  (1),  not  available/difficult 
to  get  through  (2)] 
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Utility.  The  AME  groups  mainly  focused  on  enhancing 
the  utility  of  the  ECG  System  by  modernizing  equipment, 
in  particular,  to  accept  various  types  of  digital  files.  The  four 
recommendations  from  the  MFI  group  pertained  to  digital 
transmission  modes  for  ECGs,  with  one  specifying  transmis¬ 
sion  via  the  Internet.  Recommendations  of  multiple  acceptable 
transmission  modes  from  the  domestic  group,  detailed  in  Table 
27,  included:  use  of  Internet  Protocol  (IP)  to  transmit  over 


a  high-speed  network  instead  of  a  telephone  line  (see  http:// 
transition.fcc.gov/voip/),  upload  to  an  Internet  site  using  a 
File  Transfer  Protocol  (FTP)  or  Hyper  Text  Transfer  Protocol 
(HTTP)  files,  upload  directly  to  AMGS,  fax,  or  scan  and  send 
as  an  email  attachment.  Depending  on  how  it  is  implemented, 
special  equipment/software  may  be  required  to  securely  send 
and  receive  transmissions  over  the  Internet  or  via  fax. 


Table  27.  Utility  issues/solutions  (count)  from  domestic  AMEs 


Issue/Solution  (38) 

ECG 

System 

(27) 

-  alternative  transmission  mode 
(21)  [email  (4),  fax  or  email  (3), 
fax  (2),  fax  or  Internet  (1) 

Internet:  VoIP,  iP,  FTP,  HTTP 
(6),  expand  modes  (3),  upload  to 
AMCS  (1),  digital  (1)] 

-  modernize  (6) 

MedXPress 

(11) 

-  address  challenge  to  airmen 
(11)  [naive  computer  user  (3), 
computer-naive/older  airmen  (2), 
older  airmen  (2),  no  computer 
access  (2)/mobile  app  (1),  3'^'’ 
class  user  (1)] 

100% 


30% 


E 

E 

o 

u 

<U 

V— 

O 


60% 


40% 


20% 


0% 


alternative 
transmission 
modes,  55% 


address 
chaflenge  to 
airman,  29% 


modernize, 

16% 
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In  addition,  the  domestic  AMEs  noted  challenges  MedX- 
Press  presents  to  the  airmen  who  have  limited  or  no  computer 
access  and  minimal  or  no  computer  experience,  which  may  be 
partly  due  to  generational  differences  in  the  airman  population. 

Usability.  Improving  usability  of  the  digital  systems/tools, 
for  the  most  part,  reduces  the  burden  on  the  user.  The  domes¬ 
tic  AME  recommendations,  presented  in  Table  28,  involved 


minimizing  interactions  with  the  systems/tools  through  ease  of 
access  to  information  and  navigation,  streamlining  procedures, 
and  reducing  information  overload.  The  MFI  group  s  three  us¬ 
ability  issues  focused  on  AMCS — too  time  consuming,  needs 
to  be  easier  to  correct  errors,  and  needs  to  be  easier  to  explain 
corrections  made  to  the  airmans  application. 


Table  28.  Usability  issues/solutions  (count)  from  domestic  AMEs 


mno/ 

Issue/Solution  (29) 

80%  - 

simplify, 

41% 

Online  AME 

Guide 

(8) 

-  more  user  friendly  (5)  [easier  to  use 
(2),  easier  to  navigate  (2),  easier 
access  medical  info  (1)] 

-  simplify  (3)  [quick  link/easier  access 
(2),  add  index  and  cross  referencing 
(1)] 

c  S0%  - 

0AM 

Website 

(8) 

-  simplify  (4)  [unclutter  (1),  reduce 
redundancy  (1),  info  in  single  location 
(1 ),  difficult  to  find  FAA  contact  info 
(1)] 

-  more  user  friendly  (3)  [easier  to 

o 

ro 

"D 

<U 

E 

E 

o 

u 

<u 

more  user 

navigate  (1),  easier  to  use  online  help 
(1 ),  easier  for  airman  to 
find/download  info  (1 )] 

-  larger  font/print  (1)  make  bulletin 
easier  to  read  online 

o  A0%  - 

friendly, 

38% 

AMCS 

(7) 

-  simplify  (3)  [in  general  (1),  steps  to 
print  (1),  reduce  burden  of  quarterly 
user  verification  (1)] 

-  more  user  friendly  (2) 

-  larger  font/print  (2)  [larger  (1),  MID 
and  ID#  on  printout  (1 )] 

20%  - 

0%  - 

larger  font, 

21% 

MedXPress 

(6) 

-  larger  font/print  (3) 

-  simplify  (2):  printing  steps 

-  more  user  friendly  (1)  confusing  for 
initial  airman  user 
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Adjust  Capability.  Some  of  the  existing  capabilities  require 
adjustments  to  reduce  user  frustrations,  better  support  AMEs, 
and  address  challenges  reported  by  airmen.  The  MFI  group 
had  a  single  recommendation,  which  was  to  extend  AMCS  s 
lockout  time.  The  domestic  group  indicated  similar  issues  and 
changes  to  better  meet  certification  requirements  for  completing 
and  submitting  applications  and  a  single  improvement  for  the 
online  tools  (Table  29).  Their  recommendation  to  improve  the 


search  capability  for  the  online  tools  would  provide  them  access 
to  the  right  information  on-demand.  Needed  improvements 
to  AMCS  and  MedXPress  would  better  support  requirements 
by  (a)  extending  time  before  locking  out  the  user,  (b)  easing 
password  requirements,  and  (c)  adjusting  interface  features  to 
minimize  constraints  on  AME  input  and  place  constraints  on 
airman  input.  Since  lockout  times  are  regulated  for  government 
systems,  use  of  a  warning  (pop-up  or  auditory),  as  suggested. 


Table  29.  “Adjust  capability”  issues/solutions  (count)  from  domestic  AMEs 


100% 


m% 


60% 


o  40% 


zu% 


0% 


change 
password 
require¬ 
ments,  2S% 


change 

interface, 

28% 


extend 
time  out, 
23% 


better 

search, 

23% 


Issue/Solution  (40) 

AMCS 

(19) 

-  extend  time  out  (7)  [too  short/ 
increase  (3),  frustrating  (2),  disrupts 
transmission  (1),  need  warning  (1)] 

-  change  interface  (7)  [expand 
character  limits  of  text  boxes  (3), 
reduce  need  to  go  between 
screens/keep  page  1  visible  while 
making  comments  (2),  enter  dates 
without  7”  (1 ),  hearing  field  does  not 
accept  -10  (1)] 

-  change  password  requirements 
(5):  extend  expiration  date 

MedXPress 

(12) 

-  change  password  requirements 

(6)  [reduce  challenge  for  airmen: 
easier/faster  generation  (3),  easier 
logon  (1),  password  reset  (1), 
extend  expiration  date  (1)] 

-  change  interface  (4)  [default 
demographics  to  all  CAPS  (1), 
remove/or  make  Social  Security 
number  (SSN)  field  optional  (1 ), 
remove  SAVE  button  (1),  require 
SUBMIT  to  ensure  confirmation#  (1)] 

-  extend  time  out  (2)  [too  short  (1), 
reset  with  any  action  (1)] 

Online  AME 

Guide 

(6) 

-  better  search  (6)  [hard  to  find  what 
is  needed  (3),  topic  search  would 
add  value  (2),  unable  to  locate  info 
while  airman  in  office  (1)] 

0AM 

Website 

(3) 

-  better  search  (3)  easier/direct 
access  to  info 
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may  be  the  practical  solution. 

Summary  of  Results  for  Enhancements  to  Systems/Tools. 

Tables  30  and  31  contain  summaries  of  the  191  AME  recom¬ 
mendations  for  enhancing  digital  systems/ tools.  Table  30  provides 
a  programmatic  perspective  by  showing  the  breakout  of  feedback 
within  each  category  across  the  systems/tools. 

Highlights  in  Table  30  indicate  the  system/tool  receiving 
the  majority  of  recommended  improvements  within  a  feedback 
category,  with  AMCS  receiving  the  most  overall.  Thus,  enhance¬ 
ments  to  AMCS  would  likely  provide  the  most  benefit,  especially 
in  terms  of  adding  new  capabilities,  such  as  embedding  guidance 
or  decision  aids  and  creating,  and  adjusting  existing  capabilities 
to  better  support  AME  tasks,  as  well  as  improving  the  readiness 
of  the  system  for  high  rates  of  use,  ensuring  the  integrity  of 
entered  data,  and  making  it  more  user  friendly. 

Similar  needed  improvements  to  MedXPress  would  benefit 
AME  and  airman  users,  to  include  addressing  the  challenge  it 
poses  to  airmen  with  limited  computer  access  and  experience. 
Noteworthy  is  the  absence  of  recommendations  for  improving 
the  usefulness  of  the  OAM  website,  online  AME  guide,  and 
AMCS.  To  the  contrary,  there  is  a  high  rate  of  recommendations 
to  improve  the  usefulness  of  the  ECG  system. 


Table  31  provides  the  individual  organizations  a  perspec¬ 
tive  on  needed  improvements  to  their  particular  system/ tool  by 
showing  the  distribution  of  recommendations. 

3.3.3  Changes  to  Processes  and  Policies 

The  analysis  examined  a  subset  of  170  recommendations 
involving  change  to  the  medical  certification  processes  (e.g., 
application,  examination,  submission,  and  issuance  decision) 
and  change  to  the  medical  certification  program  policies  (e.g., 
rules  governing  AMEs  and  their  authority,  airman  application 
requirements,  requirements  of  medical  certification  classes,  and 
controlled  Form  8500-8  and  paper  certificates) .  Often  there  is  not 
a  clear  line  between  process  and  policy;  thus,  that  determination 
will  be  left  to  responsible  parties  in  OAM. 

The  problems  raised  and/or  solutions  presented  involve 
changes  in  the  who,  what,  when,  where,  and  how  airman  medical 
certificates  are  issued.  In  certain  instances,  a  recommendation 
may  appear  to  belong  to  one  of  the  other  high-level  categories, 
because  its  implementation  would  ultimately  involve  the  agency 
providing  the  service,  system/ tool  support,  or  both. 


Table  30.  Breakout  of  each  feedback  category  across  systems/tools 


(Shading  indicates  highest  percent  in  each  feedback  category.) 

Add 

capability 

Reliability 

Utility 

Usability 

Adjust 

capability 

OAM  Website 

6.5% 

- 

- 

25% 

7% 

Online  AME  Guide 

6.5% 

- 

- 

25% 

15% 

AMCS 

61% 

60% 

- 

31% 

49% 

MedXPress 

26% 

25% 

26% 

19% 

29% 

ECG  System 

“ 

16% 

74%; 

- 

- 

Total  count* 

31 

44 

42 

32 

41 

*Total  count  is  the  sum  of  the  number  of  recommendations  from  both  groups;  therefore,  a  greater  contribution  from 
the  MFI  group  to  the  cumulative  percent  will  result  in  a  smaller  count  than  the  reverse. 


Table  31 .  Distribution  of  recommendations  for  each  system/tool 


(Total  count*) 

Add 

capability 

Reliability 

Utility 

Usability 

Adjust 

capability 

AMCS  (74) 

27% 

34% 

- 

14% 

26% 

MedXPress  (50) 

16% 

24% 

22% 

12% 

26% 

ECG  System  (38) 

- 

18% 

82% 

- 

- 

Online  AME  Guide  (16) 

13% 

- 

- 

50% 

38% 

OAM  Website  (13) 

15% 

- 

- 

62% 

23% 

*Total  count  is  the  sum  of  the  number  of  recommendations  from  both  groups. 
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The  five  of  the  six  actionable  feedback  categories,  defined  in 
Table  32,  align  with  stages  in  the  medical  certification  process. 
The  sixth  feedback  category  covers  FAA  rules  governing  AMEs. 
Figure  8  shows  the  distribution  of  recommendations  across  the 
categories. 

In  the  next  phase  of  the  analysis,  frequencies  were  computed 
for  the  secondary  category  assignments.  Detailed  results  are 


presented  differently  here  on  two  fronts.  First,  the  table  rows 
align  with  the  graphic,  since  the  secondary  categories  mapped 
directly  onto  the  needed  change.  Second,  AME  feedback  on 
deferral  tracking  and  standards  and  guidelines  is  respectively 
included  under  results  for  Transmission  to  the  FAA  and  Certifica¬ 
tion  Decision.  Feedback  on  certification  requirements  for  mainly 
third-class  medical  certificates  is  also  under  Certification  Decision. 


Table  32.  Actionable  feedback  categories  for  process/policy  change 


Airman  application 

-  applicant  access  to  previously  entered  medical  history 

-  change  to  items  1-20  on  Form  8500-8 

-  MedXPress  requirement 

AME  rule 

-  conduct  of  exam 

-  minimum  exams  per  year 

-  location  restriction 

Exam  appointment 

-  change  to  items  21-64  on  Form  8500-8 

-  timing  of  AME  access  to  application 

-  AME  review  of  FAA  record  on  airman  (e.g.,  previously  submitted 
medical  history,  past  exam  report,  correspondence) 

Certification  decision 

-  AME  decision  authority 

-  temporary  certificate 

Printed  certificate 

-  type  of  paper 

-  reprint  certificate 

Transmission  to  the  FAA 

-  type  of  acceptable  format  (e.g.,  digital,  paper,  fax) 

-  deadline  to  submit  supporting  documentation 

-  retrieval  of  submission  for  error  correction 

□Domestic  AMEs  (149  recommend  a  Jons)  nMFI  gtotip  (21  necortnmendations) 


transmission 
to  FAA 


airman 

ap-pficaiion 


exam 

appointment 


issuance 

decision 


printed 

certificate 


AME  rules 


Figure  8.  Distribution  of  feedback  on  process/policy  change 
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Transmission  to  thePAA.  Recommendations  mainly  focused 
on  the  need  to  electronically  transmit  completed  exams  and 
supporting  documentation.  Half  of  the  recommendations  from 
Domestic  AMEs  (Table  33)  and  two-thirds  from  the  MFI  group 
(Table  34)  supported  moving  toward  paperless  transmission.  The 
proposed  solutions  included  means  to:  Send  a  scanned  file  of  the 
completed  packet  or  scanned  supporting  documentation;  upload 


electronic  files  to  an  Internet  site;  and  either  attach  electronic 
files  to  email  or  AMCS,  or  scan  them  directly  into  AMCS.  A 
point  made  by  the  MFI  group  was  the  seemingly  redundant 
requirement  to  mail/ fax  completed  exams  and  documents  that 
had  been  electronically  transmitted.  AMEs  from  both  groups 
also  expressed  a  need  to  retrieve  submissions  in  AMCS  to  either 
correct  or  add  to  the  report. 


Table  33.  Changes  to  transmission  processes/policies  (count)  from  domestic  AMEs 


100% 


80% 


o  60  % 


AQ% 


20% 


0% 


completed 
form,  ^/% 


supporting 

document 

37% 


retrlve 

AMCS 

submission, 

13% 


extend 

deadline, 

13% 


Recommendation  (30) 

Completed 
Form  8500-8 
(11) 

-  all  electronic  (7)  [not  mailed  (4), 
send  as  a  packet  (3)] 

-  allow  paper  (2) 

-  end  redundancy-either  electronic 
or  paper  (1) 

-  ensure  medical  info  protected 
under  the  Health  Insurance 
Portability  and  Accountability 

Act  (HIPAA)  Privacy  Rule  (1) 

Supporting 

Documents 

(11) 

-  All  electronic  (9)  [as  attachment: 
email  (3),  to  form  (2),  upload  to 
Internet  site  (2),  scan  into  system 
(1),  electronically  (1)] 

-  fax  requested  info  (1) 

-  require  airman  to  send  directly 
to  AMCD  (1) 

Retrieve 

AMCS 

(4) 

-  for  corrections  (2) 

-  for  additions  (2) 

Extend 

Deadline 

(4) 

-  hold  transmission  until  required 
info  is  gathered  (1) 

-  14  days  not  enough  time  to  meet 
all  requirements  (1) 

-  not  long  enough  for  airman  to 
gather  info  (1) 

-  not  long  enough  to  lower  blood 
pressure  (1) 

Table  34.  Changes  to  transmission  processes/policies  (count)  from  the  MFI  group 


100% 


80% 


60% 


^  A0% 


20% 


0% 


completed 
form,  33% 

supporting 

document, 

33% 

ret  rive 

AMCS 

submission, 

33% 

Recommendation  (6) 

Completed 
Form  8500-8 
(2) 

-  all  electronic  (2)  [not  fax  or  mail 
(1 ),  attachment  to  form  (1 )] 

Supporting 

Documents 

(2) 

-  all  electronic  (2)  [paperless  (1), 
not  mailed  (1)] 

Retrieve 

AMCS 

Submission 

(2) 

-  for  corrections/additions  (1) 

-  for  edits  (1) 
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Deferral  Tracking.  The  domestic  and  MFI  groups  identified 
the  need  for  a  means  of  tracking  the  deferral  process  and  the 
final  decision.  The  two  recommendations  from  the  MFI  group 
suggested  the  use  of  email.  The  four  recommendations  from  the 
domestic  group  included  an  online  capability  (2),  use  of  email 
(1),  and  emphasis  on  the  criticality  of  providing  the  AME  and 
airman  a  tracking  service  to  monitor  a  deferrals  progress  (1). 


Airman  Application.  Recommendations  focused  on  the 
medical  history  page  of  Form  8500-8  (items  1-20)  and  related 
changes  to  airman  use  of  MedXPress  to  apply  for  an  airman 
medical  certificate.  The  MFI  groups  three  recommendations 
pertained  to  MedXPress:  Offer  it  as  an  option  (2)  and  remove 
the  online  forms  response  to  a  “Berlin”  address,  since  Germany  is 
united  (1).  The  majority  of  the  domestic  groups  changes  (Table 
35)  focused  on  reducing  applicant  errors  by  adjusting  wording 
and  highlighting  items  17,  18,  and  19  and  by  allowing  airmen 
to  access  and/or  reuse  input  on  past  applications. 


Table  35.  Changes  to  application  processes/policies  (count)  from  domestic  AM Es 


Recommendation  (39) 

MedXPress 

(14) 

-  optional  (11) 

-  require  airman  to  bring  printout 
to  exam  (2) 

-  add  capability  to  email 
application  to  AME  (1) 

Form  8500-8 
Items  1-20 
(12) 

-  item  19  (5)  [stress,  bold  and 
capitalize  “Visits  to  Health 

Professional  in  the  PAST  THREE 
YEARS”  (2),  change  to  read  “...past 
three  years  NOT  PREVIOUSLY 
REPORTED”  (1),  create  response 
option  for  repetitive  visits  (1),  delete 
item  (1)] 

-  item  18  (4)  [bold  "HAVE  YOU 

EVER  IN  YOUR  LIFE"  (2),  add  text 
box  to  explain  “yes”  (1), alphabetize 
(1)] 

-  item  17  (2)  [add  “PRNC”  to  17a  (1), 
reword  17b  to  reduce  errors  (1)] 

-  eliminate  response  option  of 
“Previously  Reported”  (1) 

Access  to 
Previous 
Application 
(12) 

-  auto-fill  (7) 

-  retain  data  (5)  [save  (2),  option  to 
save  or  delete  (1 ),  for  review  (1 ), 
continuous  access  (1)] 

Application 
Fee  (1) 

-  charge  airman  a  $10  fee  (1) 
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Exam  Appointment.  Recommendations,  for  the  most  part, 
focused  on  changes  that  would  reduce  AME  errors  by  provid¬ 
ing  them  with  full  information  on  the  applicant  in  their  office 
and  affect  entering  information  into  the  examiner  s  portion  of 
Form  8500-8  (items  21-64),  which  completes  the  form,  via 
AMCS.  Most  often  recommended  by  the  MFI  group  (4  of  4) 
and  domestic  AMEs  (Table  36)  was  access  to  applicants  medical 
certification  records  for  review  in  preparation  for  the  exam.  A 
new  requirement  for  applicants  to  sign  a  medical  release  form 
to  allow  their  AME  to  view  their  records  was  also  proposed. 


Certification  Decision.  Recommendations  mainly  pertained 
to  reducing  deferrals  by  expanding  the  authority  of  the  AME. 
Also  proposed  to  better  serve  airman  under  certain  circumstances 
was  expanding  AME  authority  through  the  issuance  of  temporary 
medical  certificates. 


Table  36.  Changes  to  exam  preparation  and  conduct  (count)  from  domestic  AMEs 


100%  n 

Recommendation  (32) 

80%  - 

Access 

Applicant 

Records 

(20) 

-  review  applicant’s  FAA  records 
(20)  [past  exams  (9),  past  records 
(6),  previous  medical  history  (3), 
AMCS  fiie  created  by  current  AME 
(1),  FAA  correspondence 
regarding  medical  certification  (1)] 

y,  60%  - 

c 

_o 

"D 

c 

<u 

E 

E 

n 

u 

<u 

rr 

V— 

O 

40%  - 

access 

applicant 

records, 

G3% 

Form  8500-8 
Items  21-64 
(7) 

-  remove  item  (3)  [monovision 
contacts  (1),  anai  exam  (1) 
remove  item  on  identifying  body 
marks,  because  not  consistently 
recorded  (1)] 

-  add  text  box  for  limitations  not 
listed  (1) 

-  change  eye  exam  input  to 
“pass/fail”  (1) 

-  reorder  items  to  align  with 
physical  exam  (1) 

-  ability  to  submit  addendum  (1) 

change 

Form  8500- 

S 

{items  21- 

Medical 

Release 

(2) 

-  require  applicant  to  sign 
medical  release  (2) 

20% 

64),  22% 

MedXPress 

(2) 

-  preview  application  without 
download  (2) 

signed 

medical 

release.  6% 

preview 

MedXPress, 

6% 

AMCS 

(1) 

-  display  reason  for  blocked 
exam  (1) 

0%  - 

AMCS  alert, 

_ 
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Three  of  the  four  recommendations  from  the  MFI  group 
would  expand  authority  of:  the  RFS,  the  AME  for  SI  cases 
requiring  monitoring,  and  the  senior  AME  with  advanced 
AME-Assisted  SI  (AASI)  training.  Their  fourth  recommenda¬ 
tion  was  to  issue  shorter,  temporary  medical  certification  for 


disease  monitoring.  The  domestic  group  recommended  qualified 
AMEs  be  given:  more  latitude  for  issuance  decisions,  authority 
for  AASI/SI,  opportunity  to  pursue  “super  AME”  status,  and 
authority  to  issue  a  temporary  medical  certificate  given  support¬ 
ing  documentation  while  awaiting  an  FAA  decision  (Table  37). 


Table  37.  Changes  to  decision  processes/policies  (count)  from  domestic  AMEs 


Recommendation  (22) 


Expand 

Authority 

(n=16) 

-  AME  (13)  [in  general  (6),  (AASI  (2), 
SI  (2),  AASI/SI  (1),  manage  SI 
cases  (1),  depression  screening 
(1)] 

-  senior  AME  (3)  [support  Super 

AME  program  (2),  certify  statement 
of  demonstrated  ability  (SODA)  (1)] 

Issue  Temporary  Certificate  (5)  [awaiting 
deferrai/AASI  decision(2),  with  supporting  info  from 
A/M  (1),  in  general  (1),  time  limited  for  co-morbid 
conditions  (1)] 

AME  Input  on  Deferrals  (1) 


Before  moving  to  proposed  changes  for  submissions  of  the 
examiners  report  and  supporting  documentation  to  the  FAA, 
needed  improvements  to  standards  and  guidelines  identified  by 
domestic  AMEs  are  presented. 

Standards  and  Guidelines.  Improvements  were  identified 
by  domestic  AMEs.  They  addressed  the  need  for  periodic  re¬ 
view  and  revisions,  in  general,  to  clarify  sections  and  reorganize 
the  content  so  the  process  is  easier  to  follow.  Specific  revisions 
would  involve  reviewing  certification  and  SI  criteria,  bringing 
guidelines  up-to-date  and  basing  them  on  empirical  evidence, 
as  well  as  adjusting  criteria  for  medical  tests/screening  and  is¬ 
suance.  Detailed  results  are  in  Table  38,  keeping  in  mind  that 
standards  for  airman  medical  certification,  in  terms  of  accept¬ 
able  medical  tests  and  results,  are  dictated  by  law  under  Title 
1 4  Code  of  Federal  Regulations  part  67;  thus,  standards  are  not 
easily  or  quickly  modified. 

The  majority  of  recommendations  (36  of 52, 69%)  concerned 
medical  test  requirements  and  acceptable  results.  The  remain¬ 
ing  recommendations  (16  of  52,  31%)  concerned  frequent  and 
regular  review/ rewrite  of  the  standards  and  guidelines  as  laid  out 
in  the  AME  Guide,  to  keep  it  current  and  to  provide  a  clearer 
understanding  of  the  AME  decision  making  process  within  the 
context  of  the  entire  medical  certification  process. 


Requirements  for  Class  of  Certification.  Domestic  AMEs 
joined  the  debate  on  class  medical  self-certification  in  their 
final  recommendations.  They  showed  support  for  both  sides 
of  the  debate.  Of  the  1 5  recommendations,  9  were  in  favor  of 
self-certification  ( 1  with  time  of  day  and  size  of  aircraft  restric¬ 
tions),  3  were  not  in  favor  of  eliminating  class  certification 
requirement,  and  1  was  neutral  in  pointing  out  that  with  self- 
certification,  the  FAA  needs  to  maintain  surveillance  of  the 
airmans  “adverse  driving  history.”  Three  additional  recommenda¬ 
tions  were  made— two  pertained  to  not  precluding  airmen  whose 
medical  certificate  had  been  denied,  revoked,  or  suspended,  or 
authorization  withdrawn  from  qualifying  as  a  sport  pilot,  and 
one  recommended  elimination  of  student  classification  since 
the  exam  requirements  are  the  same. 
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Table  38.  Distribution  of  recommended  improvements  to  standards  and  guidelines  (count=52)  from  domestic  AMEs 
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BP=blood  pressure;  CAD=coronary  artery  disease;  CSR=cortisol  secretion  rate;  ET0H=ethyl  alcohol;  JNC=Joint  National  Committee; 
OSA=obstructive  sleep  apnea 


Printed  Certificate.  Recommendations  focused  on  the  need 
to  reprint  a  certificate.  Half  of  the  four  recommendations  from 
the  MFI  group  pertained  to  reprinting;  one  indicated  correc¬ 
tions  were  needed.  The  domestic  AMEs’  recommendations  are 
shown  in  Table  39.  A  related  issue  pertained  to  eliminating  the 
need  for  typewriters,  for  one  MFI  AME  it  would  mean  allowing 


hand- written  certificates.  One  suggestion,  counter  to  eliminating 
typewriters,  was  to  type  the  certificate  and  later  enter  the  data 
and  transmit  the  report.  Two  recommendations  from  domestic 
AMEs  were  to  change  the  medical  certificate,  and  one  MFI 
AME  recommended  allowing  student  certificates  to  be  printed. 


Table  39.  Changes  to  printed  certificate  processes/policies  (count)  from  domestic  AMEs 


100% 


80% 


60% 


^% 


20% 


0% 


reprint 
certificate 
,  44% 


typed 

certificate 

,33% 


change 
certificate 
,  22% 


Recommendation  (9) 

Reprint 

Certificate 

(4) 

-  correct  an  error  (4) 

Typed 

Certificate 

(3) 

-  type  certificate  before  transmitting 
report  (1) 

-  eliminate  need  for  typewriter  (1) 

-  use  paper  that  accepts  ink  pen  (1) 

Change 

Certificate* 

(2) 

-  add  corrective  lens  restriction  (1) 

-  use  “chip”  on  pilot  certificate  (1) 

*Another  recommendation  was  to  remove  medical  information  from  the  front  of  authorization  letter  for  SI. 
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AME  Rules.  Table  40  shows  the  distribution  of  recom¬ 
mended  changes  to  rules  governing  AMEs  from  domestic  AMEs. 
There  were  no  recommendations  from  the  MFI  group.  Half 
of  the  recommendations  from  the  domestic  AMEs  addressed 


changes  related  to  relaxing  limitations  on  the  number  of  loca¬ 
tions,  restrictions  on  staff  in  conducting  the  exam,  restrictions 
regarding  exams  for  family  and  self,  and  the  requirement  of  1 0 
airman  exams  per  year. 


Table  40.  Changes  to  AME  rules  (count)  from  domestic  AMEs 


100%  . 

Recommendation  (16) 

-  more  locations  (3)  [automatic 
addition  of  military  reserve  FS 
reserve  location  (2),  allow  three 
(1)] 

-  staff  involvement  (2)  [easy  for 

80% 

relax 

requirement 
,  50% 

Relax 

Requirement 

(8) 

junior  partner  to  become  AME 
(1 ),  allow  PA  to  conduct  exam 
when  AME  present  (1 )] 

-  permit  family  exams  (2) 
[issuance  to  family  (1)  submit 
own  exam  (1)] 

-  remove  minimum  number  of 

annual  airman  physicals  (1) 

^60%  - 

o 

■-4-J 

"D 

r 

More 

-  immediately  notify  AME  about 
airman  complaint  (1) 

<u 

E 

Support 

-  legal  protection  under  the 

E 

o 

(3) 

Federal  Tort  Claims  Act  (1) 

u 

V 

V— 

-  redesign  as  AME-centric  (1) 

O 

40%  - 

more 

support, 

-  authorization  to  store  airman 

10% 

Record  of 
Exam 
(2) 

exam  as  electronic  medical 
record  (1) 

-  capability  to  download  list  of 
airman  exams  conducted  (1) 

record  of 

over  past  5  years 

20% 

exam,  13% 

Add 

Qualification 

(2) 

-  pilot  with  200  flight  hours  as 
pilot  in  charge  (1) 

add 

qualification 
,  13% 

-  pilot  or  military-trained  flight 
surgeon  (1) 

Add 

-  distribute  educational 

0%  - 

add 

requirement 

.6% 

Requirement 

(1) 

material  to  airmen  (1)  basic 
physiology  of  flight  brochure 
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Summary  of  Results for  Changes  to  Medical  Certification 
Processes/ Policies.  Table  4 1  summarizes  results  in  terms  of  the 
top  recommended  improvements  for  each  step  of  the  medical 
certification  process  that  the  AME  has  a  role  in,  and  for  changes 
to  standards  and  guidelines,  the  deferral  process,  and  require¬ 
ments  for  class  of  certification. 


3.4  Analysis  of  AME  Comments  Across  the  Service  Areas 

Responses  that  did  meet  the  criteria  for  a  recommendation 
were  classified  as  comments  and  were  sorted  as  negative,  neutral, 
or  positive.  The  neutral  comment  category  included  personal 
facts,  questions,  explanations,  and  hypothetical  situations.  The 
distributions  of  comments  by  group  are  shown  in  Figures  9  and 
10.  Overall,  there  were  more  positive  comments  than  neutral  or 
negative  comments,  although  domestic  AMEs  provided  com¬ 
parable  portions  of  positive  (79)  and  negative  (69)  comments. 


Table  41 .  Top  recommendation  for  the  medical  certification  process  and  AME  rules 


Airman  Application  (42)* 

31%  -  make  MedXPress  optional 

31%  -  modify  items  17-19  on  Form  8500-8 

29%  -  allow  airman  access  to  previous  application 

Exam  Appointment  (36) 

67%  -  allow  AME  access  to  applicant’s  past  records 

Standards  and  Guidelines  (52) 

35%  -  relax/expand  standards 

Issuance  Decision  (26) 

69%  -  expand  AME  authority,  with  training 

Deferral  Tracking  (6) 

50%  -  use  email 

Printed  Certificate  (14) 

46%  -  allow  corrections  and  reprinting 

Transmission  to  FAA  (36) 

56%  -  support  electronic  submission  modes 

Class  of  Certification  Requirements  (15) 

60%  -  allow  3'^''  class  self-certification 

AME  Rules  (16) 

50%  -  relax  requirements 

*(Total  count)  is  the  sum  of  the  number  of  recommendations  from  both  groups;  therefore,  a  greater  contribution 
from  the  MFI  group  to  the  cumulative  percent  will  result  in  a  smaller  count  than  the  reverse. 
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4.0  DISCUSSION 


5.0  REFERENCES 


Approximately  one  in  five  AMEs  completing  the  2012 
AME  Feedback  Survey  provided  a  written  assessment  of  the 
quality  and  delivery  of  airman  medical  certification  services 
during  the  12  months  prior  to  the  survey  by  recommending 
where  to  focus  program  improvements.  The  service  areas  they 
assessed  included  the  AMED,  AMCD,  RFS  office,  AMCS  In¬ 
ternet  system,  MedXPress,  ECG  system,  online  information  and 
publications,  AME  training,  deferral  process,  and  standards  and 
guidelines  for  medical  certification.  CAMI  used  a  content  analysis 
technique  to  translate  AME  recommendations  into  actionable 
terms  and  to  prioritize  them.  The  majority  of  recommendations 
clustered  into  three  areas  for  program  improvement:  developing 
organizational  capabilities,  enhancing  systems/ tools,  and  chang¬ 
ing  medical  certification  processes/policies.  Recommendations 
from  domestic  and  MFI  AMEs  were  similarly  distributed  and 
addressed  development  of  the  human  element  of  the  program 
nearly  twice  as  often  as  either  system/tool  enhancements  or 
process/policy  changes. 

The  higher-priority  needs  for  developing  organizational 
capabilities  would  involve  enhancing  knowledge  and  skills  of 
program  personnel  and  keeping  those  served  by  the  program 
aware  of  requirements  and  critical  information,  so  everyone  is 
working  together  to  attain  program  goals.  By  doing  so,  errors  and 
rework  are  reduced,  and  effectiveness  and  efficiency  are  gained. 

The  higher-priority  needs  for  enhancing  systems/ tools  would 
produce  gains  in  program  performance  with  stable  online  ac¬ 
cess  and  user-friendly  capabilities  that  support  end  users  of  the 
AMCS  Internet  system,  MedXPress,  ECG  system,  online  AME 
guide,  and  OAM  website  in  meeting  certification  requirements. 

The  higher-priority  needs  for  changes  to  processes/poli- 
cies  would  reduce  applicant  and  AME  errors  and  workload 
by  providing  airmen  and  their  current  AMEs  access  to  past 
medical  data/ records.  Resource  use  would  also  be  reduced  by 
transitioning  to  electronic  transmission  of  all  forms  and  sup¬ 
porting  documentation. 

Coupling  the  results  from  respondents’  written  assessments, 
which  help  identify  issues/problems  that  interfere  with  AMEs 
performing  their  designated  duties,  with  the  quantitative  sur¬ 
vey  results  provides  a  better  understanding  of  the  level  of  FAA 
support  that  AMEs  expect  and  potentially  require  as  designees. 
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